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ABSTRACT 

During the 1993-94 school year, the Office of 
Research and Evaluation of the Austin Independent School District 
(AISD) (Texas) conducted a districtwide evaluation of instructional 
technology. The evaluation consisted first of an accurate count of 
all computers in AISD schools, and then of an in-depth evaluation of 
the integrated learning systems of the Computer Curriculum 
Corporation (CCC) and Jostens Learning. The over 11,000 computers in 
the Austin schools are more than twice the amount present 3 years 
ago. Of*these, 39% are considered old. This amounts to six students 
for every one computer in the district. Gains in student achievement 
have not been significant enough to declare either of the integrated 
learning systems effective, but the gains made at some schools 
warrant their continued use. Better communication and training are 
needed to encourage use of laser disc technology and to ensure the 
effectiveness of the integrated learning systems. The district should 
add computers and plan to replace outdated models. Eighteen figures 
and two tables present evaluation findings. Four attachments provide 
supplemental information, including the principal questionnaire. 
(Contains 7 references . ) (SLD) 
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Austin Independent School District 
'Tice of Research and Evaluation 



Instructional Technology in AISD, 1993-94 

Executive Summary 

Authors: Janice Curry, Melissa Sabatino 



Program Description 



During the 1993-94 school year the 
Office of Research and Evaluation, at 
the request of the Superintendent, 
conducted a districtwide evaluation of 
instructional technology. 

The districtwide evaluation of 
instructional technology occurred in two 
stages. The first portion of the project 
consisted of obtaining an accurate count 
of all compute's in AISD schools. 

The second portion of the project 
consisted of an in-depth evaluation of 
integrated learning systems (ILS): 
Computer Curriculum Corporation 
(CCC), and Jostens Learning. During 
both portions of the project, Chapter 1 
staff were responsible for collecting 
data for Chapter 1 schools, while a 
locally funded staff member gathered 
data for non-Chapter 1 schools. 

Mqjor Findings 



1. There are 11,038 computers in 
AISD: 5,791 computers at the 66 
elementary schools, 2,461 
computers at the 15 middle/junior 
high schools, and 2,786 con^uters 
at the 1 1 high schools. This number 
is more than double the number of 
computers in i'llSD three years ago. 
(Page 4) 

2. Of AISD's 11,038 computers, 4,302 
(39%) are designated as "old." 
(Page 4) 

3. There are six students for every one 
computer in the District. At 
elementary schools there are seven 
students per computer; six students 
per computer at middle schools; and 
five students per computer at high 
schools. (Page 5) 
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Two thirds (66%) of the 1,174 
elementary teachers who responded 
to the 1993-94 ORE Coordinated 
Survey said they never use the 
Windows on Science laser disc. 
Only a small percentage (9%) of 
teachers said they use Windows on 
Science one or more times a week. 
(Page 6) 

5. At the ILS elementary* schools, the 
90 ROPE comparisons showed that 
15% of grades 2 through 5 
exceeded the predicted gain, 82% 
equaled the predicted gain, and 3 % 
were below the predicted gain on 
ITBS/NAPT scores. (Page 13) 

6. At the five ILS secondary schools, 
the ROPE comparisons showed that 
none of the grades* NAPT scores 
exceeded the predicted gain, 62% 
equaled the predicted gain, and 
37% were below the predicted gain. 
(Page 15) 

7. At the elementary and secondary 
levels, TAAS passing percentages 
for students in Jostens and CCC 
were below the District average for 
all grades and all subjects, except 
grade 4 writing, (Pagos 16-17) 

8. Based on sample data gathered for 
secondary students using CCC, 
students averaged a .02 monOi 
grade equivalent gain in reading and 
.01 month grade equivalent gain in 
mathematics for every 15 minutes 
per day of computer instruction. 
(Page 17) 

9. Observations at the elementary 
schools using ILS for one or more 
years revealed that 81% of the time 
allotted to ILS was used interacting 
with the computer on acad^jiiiic, 
technical, or procedural tasks. 
Only 5 % of the stu^.ents were 
involved in off-task behavior. 
(Page 20) 



Budget Implications 



Mandate: 
Federal and Local 

Funding Source: 

Local and external (federal and 

private) 

Implications: 

As AISD examines ways to use 
local and external monies for 
technology, the insight gained from 
the technology strategies employed 
with these curricula will be vital. 



Recommendations 



1 . Tne District should continue to add 
to the number of computers each 
year, while replacing those that are 
considered "old." 

2. Better communication and training 
are needed to «icourage teacher 
usage of the Windows on Science 
laser disc technology. 

3. At the secondary level, the CCC 
program should be implemented to 
be used 15 minutes each day as 
designed. 

4. Teacher training is needed to ensure 
the effectiveness of the ILS. 

5. Schools that use an integrated 
learning system should not rely on 
the ILS alone to improve student 
achievement. The ILS can be used 
as one of many strategies to meet 
the needs of academically diverse 
children. 
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PROGRAM EFFECTIVENESS SUMMARY 
Instructional Technology 



■ 

PROGRAM 


Rating 


Allocation 
(COST) 


Number of 
Students Served 


Cost Per 
Student 


Effect (in 
months) 


Coit per Student for 
1 month gain 
(COST/EFFECT) 


Allison 

Funding Source: Fcdcnl 
(Ciuip!er 1) 

Or»dcs: 3 - 5 

Level of Service: AU djiy/tll 
yeir 


+ 


$3^73* 

Investment cost for 
software ONLY. 


258 


$13 


R: 1.3 
M: 2.0 
Avg: 1.7 


$8 


Brooke 

Funding Source: Federal 
(Chapter 1) 

Grades: 2-5 

Level of Service: AU djiy/tU 
yetr 




$2,783* 

Investment cost for 
software ONLY. 


217 


$13 


R: 2.3 
M: 0.8 
Avg: 1.5 


$9 


Brown 

Funding Source: Jostcns 
Pilot 

Grades: 2-5 

Level of Service: AU day/aU 
year 




$3,869 (est)* 

Investment cost for 
software ONLY. 


298 


$13 


R: -1.2 
M: 1.3 
Avg: 0.05 


$260 


Jordan 

Funding Source: Federal 
(Chapter 1) 

Level of Service: AU day/aU 
year 




$2,983* 

Investment cost for 
software ONLY. 


205 


$15 


R: 1.2 
M: "0.1 
Avg: 0.6 


$25 


Norman 

Funding Source: Project 
Teach and Reach 

Grades: K - 5 

Level of Service: AU day/aU 
year 




$3,548 (est)* 

Investment cost for 
software ONLY. 


293 


$12 


R: 1.6 
M: 1.4 
Avg: 1.5 


$8 
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PROGRAM EFFECTIVENESS SUMMARY 
Instructional Technology 



PROGRAM 


Rating 


Allocation 
(COST) 


Number of 
Students Served 


Cost Per 
Student 


Effect (in 
months) 


Cost per Student for 
1 month gtin 
(COST/EFFECT) 


Oak Springs 

Fuoding Source: Project 
Teach and Reach 

GT»dcf:3-5 

Level of Service: All <Uy/»U 
year 


0 


$3^48 (est)* 

investment cost for 
software ONLY. 


160 


$22 


R: -0.7 
M: -1.7 
Avg: -1.2 




Pecan Springs 

Funding Source: Federal 
(Chapter 1) 

Grades: 3 - 5 

Level of Service: All day/all 
year 


+ 


S2,412* 

Investment cost for 
software ONLY. 


239 


$10 


R: 2.0 
M: 0.8 
Avg: 1.4 


S7 


Sims 

Funding Source: Jo«tcos 
Pilot 

Gi»dcs: Pre-K - 5 

Level of Service: All d«y/aU 
year 




$3,869 (est)* 

Investment cost for 
software ONLY. 


264 


$15 


R: 3.1 
M: -0.5 
Avg: 1.3 


$12 


Travis Heights 
Funding Source: Local 
Grades: K - 5 

Level of Service: All day /all 
year 




$3,548 (est)* 

Investment cost for 
software ONLY, 


694 


$5 


R: 0.3 
M: 0.1 
Ave- 0 2 


$25 


Winn 

Funding Source: Project 
Teach and Reach 

Grades: K. 2 - 4 

Level of Service: All day /all 
year * 




$3»548 (est)* 

Investment cost for 
software ONLY. 


281 


$13 


R: -1.6 
M: 2.7 
Avg: 0.6 


$22 
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PROGRAM EFFECTIVENESS SUMMARY 
Instructional Technology 





PROGRAM 


Rating 


Allocation 
(COST) 


Number of 
Students Served 


Cost Per 
Student 


Effect (in 
months) 


Cast ner Student for 
1 month gain 
(COST/EFFECT) 




Fulmorc 

Funding Source: Extemtl 
(JTPA) 

Grades: 7 - 8 

Level of Service: AU 
day /2nd semester 


0 


$4,458** 

Investment cost for 
hardware arid 
software. 


110 


$41 


R: -3.8 
M: -3.0 
Avg! -3.4 






Pearce 

Funding Source: External 
(JTPA) 

Grades: 7-8 

Level of Service: NoC Used 
in 1993-94 


n/a 


$4,458** 

Investment cost for 
hardware and 
software. 


0 


SO 


R: n/a 
M: n/a 

r^Vg. 11/ a 


n/a 


Porter 

Funding Source: External 
(JTPA) 

Grades: 7 - 8 

Level of Service: AU day /all 
year 


0 


$4,458** 

Investment cost for 
hardware and 
so iiware. 


106 


$42 


R: -4.8 
M: -2.5 
Avp' -3 7 




Crockett 

Funding Source: External 
(JTPA) 

Grades: 9 

Level of Service: All 
Day /2 nd semester 


0 


$4,458** 

Investment cost for 
hardware and 
software. 


57 


$78 


R: -4.0 
M: -8.0 
Avg: -6.0 


- 


Robbins 

Funding Source: External 
(JTPA) 

Grades: 9 - 10 

Level of Service: AU day/all 
year 




$4,453** 

Investment cost for 
hardware and 
software. 


140 


$32 


R: 5.0 
M: -1.0 
Avg: 2.0 


$39 
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PROGRAM EFFECTIVENESS SUMMARY 
Instructional Technology 



PROGRAM 


Rating 


AUocttion 
(COST) 


Number of 
Students Served 


Cost Per 
Student 


Effect (in 
months) 


Cost per Student for | 
1 mom^? gain ; 
(COST/teFFECT) \ 


Reagan High School 

Funding Source: Extcnul 
(JTPA) 

Gfadci: 9-12 

Level of Service: Not Used 
duiiiic 1993-94 


n/a 


$4,458** 

investinent cost for 
l^rdwtrc and 
software. 


0 


$0 


R: n/a 
M: n/a 
Avg: n/a 




Travis High School 

Funding Source: Hxtenul 
(JTPA) 

Ondc*: 10 - 12 

Level of Service: All dAy/aU 
year 


0 


, $4,458** 

investirient cost for 
hardware and 
software: 


100 


$45 


R: 1.6 
M: -7.9 
Avg: -4.8 


*• 



Notes: ♦ The cost for the elementary CCC and Jostens laboratories includes software ONLY. The 
initial software investment was multiplied by 6.7% according to TEA guidelines concerning 
depreciation (TEA Procedure: SPG-702). 



The cost for th^ CCC laboratories at the seven secondary schools involved an initial investment 
of $455,711 for hardware and software. In 1993-94 AISD spent $10,(XX) 1^ buy the seven 
laboratories from the Private Industry Council bringing the total to $465,711. This cost was 
multiplied by 6,7% aixording to TEA guidelines for depreciation and divided by 7 to arrive at 
the allocation cost for each secondary school. 



Cott is the expeiue over the regular District per-itudent 
expenditure of $4,000. 

0 No cost or imnimal cost 

$ Indirect costs and overhead, but do f>cparttc budget 
$$ Some direct costs, but uiuler $500 f^cr ttudent 
$$$ Major direct costs for teachrn, rtaff, and/or 

e(]uipment in the range of $500 per student or more. 
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Rating ii expressed as contributing to any of the 
five AISD strategic objectives 

+ Poiilive, needs to be kept and expanded 
0 Not significant, needs to be iniprovcd and 
modified 

Negative, needs major nKxlification or 
replacement 
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Initnictiooil Tcchnoloiy in AISD, 

CONCLUSIONS AND RECOMMENDATIONS 

With more and more schools purchasing computers to improve student achievement the District 
realized a need to find what computers were in the schools. There was no thorough list of the 
numbers and types of computers at the campuses. During the 1993-94 school y^, the Office of 
Research and Evaluation (ORE) conducted a districtwide computer mventory. The results showed 
that AISD has more than doubled the number of computers available tx) students at elementary ^^^^^^^^^^ 
Tchool and high school campuses from 5,000 in 1990-91 to 11,038 in 1993-94. Of AISD s current 
nm^berS computers, 39% (4,302) are designated as "old." Many of the schools contmue to use 
Texas Instruments and Apple computers in the classroom, while most schools use the newer 
computers in a lab setting. 

In order to ascertain the availability of computers to students, the ratio of students to computer was 
calculated for the District and by the type of school. The ratio of students per computer is six 
tudenTrrr eve^^ one computer in the District. At elementary schools the ratio is seven to one at 
rnWdle schools the ratio decreases to six to one, and high schools have the lowest ratio, at five to one. 

During the 1990-91 school year, AISD adopted Windows on Science for its science curriculum in 
Serg^en through grade 5. On the 1993-94 ORE Coordinated survey ^'^^^t^^^f^^^^^^^^ 
asked how often they used the Windows on Science laser discs. Two out of thr^ ^^^^^^^^^^ 
1 174 elementary teachers who responded to the survey said they never use the Windows on 
Science Zlr disc technology. Only a small percentage (9%) of teachers said they use Windows 
on Science one or more times a week. 

There is a growing interest in the use of integrated learning systems QLS) as districts are tiying rn^y 
nrive melds of instruction to reach children of varying achievement levels. Twenty-two AISD 
e emen^ry and secondary schools used an ILS with the aim of improving student achievement m 
1993-94 Computer Curriculum Corporation (CCC) and Jostens Learning were the major integrated 
rning systems used in the District. For the purpose of this evaluation, only tiie ten e emen^y and 
five secondary schools which had used ILS for one year or more were studied for effect on student 
achievement. The results, after one year, showed that: 

. Using ROPE (Report On Program Effectiveness) at the ten elementary schools, 15% of grades 
2 through 5 students exceeded the predicted gain, 82% equaled the predicted gain, and 3% 
were below the predicted gain on the Norm-referenced Assessment Program for Texas 
(NAPT) test. 

. At the five secondary schools, none of the grades exceeded the predicted gain, 62% equaled 
the predicted gain, and 37% were below the predicted gain on the NAPT. 

There were only seven grade levels at six different elementary schools that made statistically 
significant achievement gains, leading to the inference that the program had a positive impact 
on achievement at those schools. 

At all levels TAAS passing percentages for students using Jostens and CCC were below the 
District average for all grades and all subjects, except grade 4 writing. 

Page 1 



10 



93.06 



Ifwtnictioiial Tcchoolofy m AISD. 1993-94 



• Observations at the elementary schools using ILS for one or more years revealed that 81% of 
the time allotted to ILS was used interacting with the computer on academic, technical, or 
procedural tasks. Only 5% of the students were involved in off-task behavior. 

While the ILS programs cannot be called overwhelmingly effective, there have been some gains in 
student achievement. After the first year, many of the schools should reassess the usage of their ILS 
and search for methods of improvement. 



Recommendations: 

1) The District should continue to add each year to the number of computers which are available for 
student use, while replacing diose that are considered "old." The implementation of Central 
Receiving during the summer of 1994 will make the task of tracking campus technology much 
easier, 

2) Better communication and training are needed to encourage teacher usage of the Windows on 
Science laser disc technology. 

3) At the secondary level, the CCC program should be implemented to be used 15 minutes each day 
as designed. 

4) Teacher training is needed to ensure the effectiveness of the ILS. 

5) Schools that use an integrated learning system should not rely on the ILS alone to improve student 
achievement. The ILS can be used as one of many strategies to meet the needs of academically 
diverse children. 



ERLC 
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INTRODUCTION 



The Austin Independent School District (AISD) has a long history of applying computer technology to 
instruction, beginning in earnest with the District's "computer initiative" in the early 1980's. Since 
the mid-1980's, with the assistance of Chapter 1 and Chapter 2 monies, AISD has installed many 
different computer software systems - e.g., Wic^^t, Writing to Read, Computer Curriculum 
Corporation (CCC), and Jostens at the elementary level, and Texas Learning Technology Group, 
Technology Learning Center, and CCC at the secondary level. Major sources of funding for 
instructional technology in AISD include grants from IBM, Apple, the Texas Education Agency, 
Pepsi Corporation, RJR Nabisco, District funds, and local Parent Teacher Association (PTA) 
resources. In 1993-94 AISD utilized computers for instructional purposes extensively throughout the 
District. 

During the 1993-94 school year, the Office of Research and Evaluation (ORE), at the request of the 
Superintendent, conducted a districtwide evaluation of instructional technology. Local resources were 
allocated to evaluating instructional technology in non-Chapter 1 schools, and Chapter 1 resources 
were allocated to evaluating instructional technology in Chapter 1 schools. ORE staff determined 
what computers were in use in the District, and performed an in-depth evaluation of the effectiveness 
of integrated learning systems (CCC and Jostens) at schools which have utilized these systems for one 
or more years. 

This report on instructional technology is divided into two parts. The first section reports the findings 
of the districtwide technology inventory. The second section takes an in-depth look at integrated 
learning systems (CCC and Jostens) used in AISD schools, and their effect on achievement. 



INSTRUCTIONAL TECHNOLOGY IN THE SCHOOLS 



METHOl OLOGY 

In a coordinated effort involving ORE, the Department of Management Information, and Instructional 
Technology, AISD attempted to obtain an accurate count of all computers in the District. 

During spring 1994, Chapter 1 staff and several consultants collected information from all Chapter 1 
schools, and one local staff member and several consultants collected the same information from all 
other schools. Instructions to those completing the inventory were to count any computer AISD is 
responsible for repairing. Only computers in working order or with a work order pending were 
counted. The information was collected on inventory sheets designed by Instructional Technology. 
The information collected consisted of the type and the number of models available at each school, 
and whether the technology was in the classroom, laboratory, or administrative offices. See 
Attachment A for an example of the inventory sheets. A group of individuals trained for this task 
visited every room in every school in the District to inventory computers. The District administration 
building was not included in this inventory. 
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NUMBER OF COMPUTERS DISTRICTWIDE 

There are 11,038 computers districtwide. Figure 1 shows that there are 2,786 computers at the 
District*s eleven high schools, 2,461 computers at the District's fifteen middle and junior high 
schools, and 5,791 computers at the District's 66 elementary schools. Figure 1 also shows whether 
the computers were in the classroom, a computer laboratory, or administ ative offices. See 
Attachment B for a breakdown of the location of computers by school. 



FIGURE 1 

NUMBER OF SCHOOL COMPUTERS BY LOCATION, MAY 1994 





Classroom 


Laboratory 


Administrative 


Total ' 


Elementary 
Schools 


3,042 


2,405 


345 


5,791 


Middle/Junior 
High Schools 


939 


1,236 


284 


2,461 


High Schools 


1,065 


1,228 


493 


2,786 


AISD TOTAL 


\ 5,046 


4,869 


1,124 


11,038 



The last technology inventory, performed in 1990-91, showed approximately 5,000 computers in the 
District, which is less than half of the 11,038 inventoried in 1993-94. The doubling of computers in 
less than three years has been supported from several sources. The Superintendent has led a 
movement for computers in the middle schools with the 1993-94 purchase of a writing lab for every 
middle and junior high school. On the spring 1994 ORE Coordinated Survey, 57% of principals said 
that at their school the largest portion of funds to purchase computers came from District funds. 
Twenty percent of principals said tliat the largest portion of funds to purchase computers at their 
school came from federal funds. The remaining 20% of principals said their school's major funding 
source for computers was PTA fur,ds, private company grants, and other donations. See Instructional 
Technology in AISD Technical Report (ORE i ublication No. 93. E) for a complete breakdown of the 
ORE Coordinated Survey responses. 



NUMBER OF "OLD" VS. "NEW" COMPUTERS IN THE DISTRICT 

Since it is often difficult to obtain software and repair parts for older machines, it is important to 
know the age of the computers used in AISD. The age of the District's computers was determined 
using the model type of the machine. Of the District's 11,038 machines, 4,302 (39%) are 
designated as "old." An "old" machine is defined as any IBM or IBM-compatible machine with less 
than a 286 based microprocessor, a Texas Instruments machine, or any Apple machine. See Figure 2 
for the numbers and percentages of "old" and "new" computers in AISD. 
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nGURE2 

NUMBER AND PERCENT OF "OLD" VS. "NEW" SCHOOL COMPUTERS, MAY 1994 





Total 


"Old" Machines 


"New" Machines 
Total Minus "Old" 


Elementary 
Schools 


5,791 


1,819 (31%) 


3,972 (69%) 


Middle/Junior 
High Schools 


2,461 


1,123 (46%) 


1,338 (54%) 


High Schools 


2,786 


1,360 (49%) 


1,426(51%) 


AISD TOTAL 


11,038 


4,302 (39%) 


6,736 (61%) 



RATIO OF STUDENTS TO COMPUTERS 

In order to get an idea about the availability of computers to students, the ratio of students per 
computer was calculated for the District and by the type of school. The ratio of students to 
computers is six students for every computer in the District. At the elementary school level there 
are seven sttidents for every computer. At middle school that ratio decreases to six students for every 
computer. High school has the lowest ratio, five students for every computer. 

When the number of "old" computers is removed from the analysis and tlie ratio of students to 
computers is recalculated, the number of students to computers rises markedly. When the number 
of "old" computers is removed from the calculation, there are 10 students for every "new" 
computer districtwide. At the elementary and middle school levels there are 10 students for every 
computer. High school has the highest ratio with 11 students for every computer. See Figure 3 for 
the number of students per computer in the District. 



FIGUR.E3 

NUMBER OF STUDENTS PER COMPUTER IN THE DISTRICT, MAY 1994 



Elementary 
Schools 



Middle/Junior 
High Schools 



"Total" 
Number of 
Computers 



5.791 



2,461 



High Sch ools 
AISD 



2,786 
11,038 



Number of 

"Old" 
Computers 



1,819 



1,123 



1,360 
4,302 



"Total" 

Minus 

II 



Old" 



3,972 



Number of 
Students 



1,338 



1,426 
6,736 



39,872 



14,580 



15,313 
69,765 



Students Per 
Computer 
"Total" 



Students Per 
Computer "Total" 
Minus "Old" 



10 



11 



11_ 
10 
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LASER DISC PLAYERS 

During the 1990-91 school year, AISD adopted Windows on Science for its science curriculum in 
kindergarten through grade 5. Windom on Science is a laser disc set. Every elementary school in 
the District was allocated a set of laser discs players. The AISD adoption was 70% for textbooks and 
30% for optical discs {Windows on Science), The allocation formula was one laser disc player for 
each grade at every school. If a grade had more than four teachers, an additional laser disc player 
was purchased for that grade. 

On the 1993-94 ORE Coordinated survey, elementaiy teachers were asked how often they used the 
Windows on Science laser discs. Two out of three (66%) of the 1,174 elementary teachers who 
responded to the survey said they never use the Windows on Science laser disc technology. Only 
a small percentage (9%) of teachers said they use Windows on Science one or more times a week. 

Elementary teachers were also asked whether Windows on Science was an effective method of science 
instruction. One \\\ three (31%) agreed or strongly agreed that Windows on Science was an 
effective method of science instruction. Over half (55%) of the 1,120 teachers responding to the 
question were neutral about whether it was effective. It is not clear if this response is due to the 
fact that these teachers were displeased with the quality of Windows on Science or that they lacked 
exposure to the laser disc technology. 



SUMMARY 

As of May 1994, there were 11,038 computers in use in AISD elementary, middle/junior high 
schools, and high schools. This number is more than double the number of computers in the District 
since the last technology inventory in 1990-91. Of this number, 4,302 (39%) are considered to be 
"old" machines. The ratio of students per computer is six students per computer in the District. 
When the number of "old" computers is removed from the calculation, there are 10 students for every 
"new" computer districtwide. 

The Windows of Science laser discs is a type of technology supplied to AISD elementary schools to 
supplement the science curriculum in kindergarten through grade 5. Two out of three (66%) of the 
1,174 elementary teachers who responded to the 1994 ORE Coordinated Survey said they never use 
Windows on Science laser disc technology. It is not clear if this response is due to the fact that these 
teachers were displeased with the quality of Windows on Science or that they lacked exposure to the 
laser disc technology. 
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THE EFFECTIVENESS OF I^JTEGRATED LEARNING SYSTEMS 



With the current emphasis in education on heterogeneous grouping and inclusion of all students, 
schools are challenged to meet the instructional needs of students who are at different skill levels 
within the same classroom. Many schools have begun to use integrated learning systems QLS) as one 
of the ways to help meet the needs of low-achieving students. The ILS allows for different levels of 
difficulty to be targeted at different students through the computer networking device. (Nichols, 1992) 

WHAT IS AN INTEGRATED LEARNING SYSTEM? 

An integrated learning system QLS) is a computer system that provides instruction in several subject 
areas and practice problems covering a multiple-year curriculum sequence. Software is housed on a 
central file server computer linked in an electronic network to 15 to 30 student computers. Specific 
lessons are automatically loaded into each student's computer when that student "logs in" based on a 
continuous assessment of that student's previous accomplishments and current learning needs. The 
ILS includes a wide range of courseware with a sophisticated management system that can be tailored 
to District objectives. These systems use computers to diagnose, reinforce, and enhance learning. 
The systems monitor student achievement and provide documentation of student improvement (May, 
1991). 

RESEARCH ON INTEGRATED LEARNING SYSTEMS 

Some issues considered in studies on ILS systems include student achievement, curriculum 
integration, teacher's role, financial considerations, staff training, and administrative support. 
Research is not conclusive about the effect on achievement for students using an ILS, but teachers, 
parents and principals have stated that the ILS had positive effects on children's learning. A study 
by May in 1991, which compared three ILSs (CCC, Jostens, and Ideal), showed that CCC was more 
expensive, but students made more academic gains than with Jostens. 

Research findings were supported in a 1990 paper by Alifrangis entitled An Integrated Learning 
System in an Elementary School: Implementation, Attitudes, and Results. Her study indicated that 
computer programs whose management is not an additional burden on the.classroom teacher, match 
the local curricula, and address learning theory research can be implemented successfully. 

The importance of a full-time laboratory operator was repeatedly noted in a study of the Albuquerque, 
New Mexico public schools (Resta & Rost, 1986). Tlie authors found that the computer-assisted 
instruction programs were underutilized during their first year, and that the impact on mathematics 
achievement was generally greater than on reading. 

There is a growing interest in these systems as school districts try many alternative methods of 
instruction to reach academically diverse children. Many of the studies are presented by vendors. A 
majority of the implementations seem to have been effective, although witliout knowing more about 
each setting, one cannot be sure that results were not due to other factors such as changes in 
educational services coincident with the implementation of the ILS (Becker, 1990). 
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INTEGRATED LEARNING SYSTEMS IN AISD 

The two major integrated learning systems used in the AISD were Computer Curriculum Corporation 
(CCC) and Jostens Learning. They are similar, in that they both offer curriculum in the areas of 
mathematics, science, reading, and language skills. Both systems can generate reports to be used by 
teachers and principals to evaluate progress. CCC and Jostens are closed systems because "the 
system" evaluates and diagnoses the progress and places each student on a daily basis. Jostens has 
courseware correlating to the Texas Assessment of Academic Skills (TAAS) objectives for Texas. 
CCC will be introducing similar courseware in fall 1994. 

There were 14 Chapter 1 schools and eight non-Chapter 1 schools in AISD which worked to boost 
achievement through the use of an ILS. Of the schools using ILS, 19 (86%) used CCC and 3 (14%) 
used Jostens. CCC was utilized at both elementary and secondary levels, while Jostens was used only 
at the elementary level. Among the 22 campuses using an ILS, there were 15 elementary schools, 3 
middle schools, and 4 high schools. 

Chapter 1 ILS Elementary Schools 

The 14 Chapter 1 schools utilizing ILS were Allison, Harrington, Brooke, Brown, Houston, Jordan, 
Linder, Norman, Oak Springs, Pecan Springs, Sims, Winn, Wooldridge, and Wooten. Five of these 
schools (Barrington, Houston, Linder, Wooldridge, and Wooten) were not evaluated because their 
systems had been in use less than one school year. For the purpose of this evaluation, only the nine 
Chapter 1 campuses which had operated with CCC or Jostens for one year or more were studied 
(Allison, Brooke, Brown, Jordan, Norman, Oak Springs, Pecan Springs, Sims, and Winn). 

An ILS may be used in a lab setting or as a distributive network in the classroom. Only two of the 
schools use the distributive network (Jordan and Wooldridge) while two other schools (Norman and 
Linder) hope to add that capability for 1994-95. 

Chapter 1 ILS Grades Served 

Each campus had special needs to consider when deciding which students to target to use the ILS. 
Some schools used the 1992-93 TAAS and NAPT scores to pinpoint areas of greatest need, while 
other schools focused on specific grade levels. Two schools (Sims and Norman) chose to serve the 
entire student population, while two schools (Linder and Wooten) served only the Chapter 1, at-risk, 
and below grade-level students. 

In May 1994, questionnaires were sent to 15 elementary principals who currently have an ILS; nine 
responded for a 60% return rate. They were asked to respond to the question. What was the initial 
goal for the implementation of CCC or Jostens at your campus? See Attachment C for a copy of the 
Principal Questionnaire. Responses from principals included the following goals: 

• To improve TAAS scores; 

• To improve student achievement in the areas of reading and mathematics; 

• To serve Chapter 1, at-risk, or below-grade-level students; 
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• To provide basic skills to students lacking them, while providing enrichment for more 
advanced students; and, 

• To increase the use of technology as an instructional tool. 

Across all schools, the grade level most targeted for ILS usage was grade 4 with an average of 142 
minutes of usage each week, followed by grade 5 with 135 minutes per week, and grade 3 with 133 
minutes each week. This time allotment is understandable because students begm takmg the TAAS 
test in the grade 3. Lesser amounts of time were allotted to lower grades, with only one school 
(Sims) including pre-K classes. One 2nd-grade class at Winn gave up its outside time to work in the 
ILS lab. Figure 4 shows the amount of time that students at the nine Chapter 1 campuses used the 
ILS in a typical week. 

nGURE4 
CHAPTER 1 ILS USAGE, 
MINUTES PER WEEK BY GRADE AND SCHOOL ^^^^ 



SCHOOL 



Allison 



Brooke 



Brown (Jostens) 



Jordan* 



Norman 



Oak Springs 



Pecan Springs 



Pre-K 



K 



80 



1st. 



120 



2nd. 



150 



70 



80 



3rd 



120 



150 



105 



100 



120 



150 



150 



120 



150 



175 



100 



80 



150 



150 



5th:-:v.. 



120 



ISO 



105 



100 



120 



150 



150 



Sims 
(Jostens) 



30 



25 



30 



35 



150 



205 



Winn 



45 



Avg. Minutes Per 
Week 



30 



50 



75 



♦*150 



150 



150 



97 



133 



142 



DiiUibutive network 
One Kcond grade only 



185 



135 



Nnn-Chaotfr 1 ILS Elem entary Schools 

The only non-Chapter 1 elementary school in the District to utilize CCC for longer than one year was 
Travis Heights Elementary. Travis Heights has had a CCC laboratory for two years and utilizes the 
lab at every grade. Pre-K and kindergarten students visit the CCC lab once a week, while students at 
grades 1 through 5 rotate through the lab at half-hour intervals the other four days. 
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Secondary ILS Schools 

The District has four high schools (Crockett, Reagan, Robbins, and Travis) and three middle schools 
(Fulmore, Pearce, and Porter) using CCC in a lab setting. In 1991-92, these seven schools entered 
into a contract with the federally funded Job Training Partnership Act (JTPA) to provide job training 
skills to local high school and middle school students. Each school was allowed to use the lab as it 
deemed necessary as long as 50% of the lab users were JTPA-approved students. To become JTPA 
approved, each student and his or her family had to complete a questionnaire concerning family 
finances, employment history, and credit history. 

In previous years, Communities-in-Schools personnel helped complete the paperwork to get students 
JTPA approved, in order for the school to use the lab. However, in 1993-94 no monies were 
budgeted for personnel to get students JTPA approved, and tlie job fell to school personnel. This lack 
of additional personnel to get students JTPA approved led to little or no usage of these computers in 
many high schools and middle schools. During the 1993-94 school year, only Travis High School, 
Robbins High School, and Porter Middle School used the computers for the entire year. Crockett 
High School and Fulmore Middle School were unable to use the machines for the entire year because 
of their inability to get the students JTPA approved. The CCC laboratories at Crockett and Fulmore 
were used the second semester, after students were approved. Reagan High School and Pearce 
Middle School did not use the CCC laboratories during the 1993-94 school year because of several 
hardware problems. 

Secondary Grades Served 

Each school is using the CCC lab in a different manner. Travis High School is using the CCC lab to 
help grades 10, 11, and 12 students who have not passed the exit-level TAAS. Robbins is using the 
CCC lab to help grades 9 and 10 students who enter the school to advance to the high school level 
before entering the high school curriculum. These students stay in the lab environment until their 
computer score shows they have maintained a grade equivalent score of 8.5. Fulmore Middle School 
and Crockett High School are using the CCC program to help students in grades 7 and 8, and grade 
9, respectively, who have fallen behind in reading. Porter Middle School is using the CCC program 
to prepare grades 7 and 8 students for the TAAS test. 
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ACfflEVEMENT AND PROMOTION DATA 



Report On Program Effectiveness 

The schools studied differ on many factors, and to compare their achievement scores directly could be 
misleading The Report On Program Effectiveness (ROPE) provides a more accurate mterschool 
comparison of achievement results. ROPE gives information on how each school's students perform 
on standardized tests (NAPT/ITBS) from one year to the next compared to similar students across the 
District The report combines the individual scores of each student in a school program. ROPE 
adjusts the scores for factors out of the school's control (i.e., sex, previous achievement level, 
ethnicity, income level, and age in grade) before making the comparison. 

ROPE compares students' actual scores with a predicted score for each student. The difference, 
called a residual is an indication of how far above or below prediction a student performed on a test 
compared to students with similar characteristics. The residuals of all students in a program are 
combined to create a program's ROPE scores. 

Three ROPE results are pcssible: exceeded predicted gain, achieved predicted gain, and below 
predicted gain. A score of achieved predicted gain indicates that an additional program (the ILS) had 
no effect on student achievement above and beyond everyday classroom teaching. If the results 
exceed predicted gain, one can conclude that tiie program had a significant impact on student 
achievement. If the results are continually below predicted gain, the program may need to be 
reexamined. A score exceeding or below predicted gain is based on a statistical test to determine if 
the residual is significantly different from zero. 

The following section presents the ROPE scores for the schools studied. ROPE generates scores only 
on students who have valid standardized test scores for the previous year; therefore, kindergarten and 
grade 1 students are not included in tl-.e results. 

ROPE Scores and Comparisons 
Elementary Schools 

Figure 5 displays ROPE scores by test area (reading, mathematics, and language) for the 10 
elementary ILS schools studied. Grades 4 and 5 showed the most scores (5 ^ch) which exceeded the 
predicted gain. The analysis uses ROSE scores to evaluate how a school performed in relation to the 
predicted gain for that school. 

Ten comparisons were a possible 10 scores-two each in grades 2 and 3, and three each in grades 4 
and 5 Allison, Jordan, Oak Springs, Pecan Springs, and Winn had eight comparisons because their 
grade 2 students did not take ITBS. The results of the program effectiveness analyses are found m 
Figure 6. 
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FIGURES 

PROGRAM EFFECTIVENESS BY TEST AREA, ILS ELEMENTARY SCHOOLS, 1993-94 
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♦ Grades 3, 4, and 5 at Brooke; grade 5 at Brown; grades 2. 3, and 5 at Norman; grades 2, 3, and 5 at Sims; and grade 2 at 
Travis Heighu did not have the required number of 25 studenU with valid NAPT scores for two consecutive year* to test 
sUtislicai significance. These resulu should be interpreted with caution. 



KEY 

10 Achieved Predicted Gain - Below Predicted Gain 

+ Exceeded Predicted Gain n/a Test not Given 



I 



Figure 6 shows a total of 90 ROPE scores for the 10 elementary schools using an ILS, Fifteen 
percent of the grade 2 through 5 scores exceeded the predicted gain, 82% equaled the predicted gain, 
and 3% were below the predicted gain. The following grades and schools made gains significant 
enough to assume that the program had a positive impact on achievement: 

Reading Mathematics 

Sims - Grade 4 Allison - Grade 4 

Pecan Springs - Grade 5 Brooke - Grade 2 

Pecan Springs - Grade 4 

Winn - Grade 2 

Winn ~ Grade 3 
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Exceed 
Predicted 



Below 
Prcdiclcd 



nGURE6 

COMPARISON OF ROPE SCORES, ILS ELEMENTARY SCHOOLS, 1993-94 



Allison 



1 (13%) 



0 (0%) 



Brooke 



Brown 



4 (40%) 



0 (0%) 



1 (10%) 



0 (0%) 



Jordan 



0 (0%) 
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Norman 



2 (20%) 



0(0%) 



Oak 

Springs 



0 (0%) 
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Pecan 
Springs 



2 (25%) 



1 (13%) 



Sims 



1 (10%) 



1 (10%) 



Travis 
Hts, 



0 (0%) 



0 (0%) 



Winn 



2 (25%) 



1 (13%) 



5 (63%) 



Equal 
Predicted 



7 (88%) 



6 (60%) 



9 (90%) 



8 (100%) 



8 (80%) 



8 (100%) 



5 (63%) 



8 (80%) 



10 (100%) 



In addition to examining achievement data on each elementary school using an ILS, two other types 
of comparisons were made: 1) each ILS program (CCC and Jostens) was looked at individually, and 
2) both programs were looked at together, ROPE was used to compare program (Jostens and CCC) 
students. Tlie ITBS/NAPT scores from spring 1994 were compared to predicted levels of 
achievement. The results of the analysis based on 10 comparisons was: 1) CCC students achieved 
predicted gains in all comparisons, 2) Jostens students achieved predicted gains in 8 out of 10, 
exceeded predicted gains in one comparison, and were below predicted gains in one comparison. IL! 
program students' scores are noted in Figure 7. 



HGURE? 

PROGRAM EFFECTIVENESS FOR ITBS/NAPT 
ELEMENTARY CCC, JOSTENS, AND TOTAL ILS 



PROGRAM 


Reading 


Mathematics 
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Secondary Schools 

Figures 8 and 9 display ROPE results for the secondary ILS schools studied. Reagan High School 
and Pearce Middle School are not included because no students were ser\'ed in 1993-94 due to 
computer hardware problems. Further analysis of the scores is provided in Figure 10. 

FIGURES 

PROGRAM EFFECTIVENESS BY TEST AREA, ILS fflGH SCHOOLS, 1993-94 





Crockett* 


Robbins* 


Travis* 


Grade 9 








Reading 


0 


0 


n/a 


Mathematica 






n/a 


Language 


0 


0 


n/a 


Grade 10 








Reading 


n/a 


n/a 


0 


Mathematica 


n/a 


n/a 


0 


Language 


n/a 


n/a 


0 


Grade 1 1 








Reading 


n/a 


n/a 




Mathematics 


n/a 


n/a 




Language 


n/a 


n/a 





♦ Grade 9 at Robbins and gradea 10 and 1 1 at Travia did not have the 
required number of 25 student* with valid NAPT scores for two 
consecutive yean to teat statistical significance. These results 
should be interpreted with caution. 



FIGURE 9 

PROGRAM EFFECTIVENESS BY TEST AREA, ILS MIDDLE SCHOOLS, 1993-94 
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♦ Grade 8 at Fulmore and grade 7 at Porter did not have the 
required number of 25 studenU with valid NAPT scorea for 
two consecutive years to test statistical significance. 
Thccc results should be interpreted with caution. 



KEY 



10 Achieved Predicted Gain - Below Predicted Gain I 
+ Exceeded Predicted Gain n/a Test not Given I 
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FIGURE 10 

COMPARISON OF ROPE SCORES, ILS SECONDARY SCHOOLS, 1993-94 
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Robbim 
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Exceed 
Predicted 


0(0%) 
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0 (0%) 


Equal 
Predicted 


2(67%) 


2(67%) 


3 (50%) 


4 (67%) 


4(67%) 


Below 
Predicted 


1 (33%) 


1 (33%) 


3 (50%) 


2 (33%) 


2(33%) 



At the five secondary schools using an ILS, none of the grades exceeded the predicted gain, 62% 
equaled the predicted gain, and 37% were below the predicted gain. 



ROPE Results 

In conclusion, the ROPE results show that ILS technology may have had a slightly positive 
impact on student achievement at the elementary level, while having little impact at the 
secondary level* In general, grades 4 and 5 showed the most improvement, with each grade 
exceeding predicted gains in 5 of 30 ROPE comparisons. 



TAAS Comparisons 

The Texas Assessment of Academic Skills (TAAS) is a criterion-referenced test which is designed to 
measure a well-defined set of skills and to reference students* scores to a mastery criterion. The 
skills axe a subset of the Essential Elements adopted by the State Board of Education. TAAS reading 
and mathematics tests were given in spring 1994 to grades 3 through 10 (exit level), while TAAS 
writing was given to grades 4, 8, and 10 (exit level). 

Elementary ILS Programs 

TAAS comparisons were made between the two ILS programs to examine possible differences in the 
TAAS passing rates of their participating students. The data in Figure 1 1 show a greater percentage 
of the Jostens students passed all tests taken in grades 3 and 5 than CCC students, while grade 4 
students using CCC had a higher passing percentage than Jostens* students. 

Overall, AISD passing percentages for all subjects and all grades were higher than either the CCC or 
Jostens percentages, except at grade 4 writing. Grade 4 writing scores were higher for Jostens (61 % 
passing) and CCC (68% passing) than the AISD average passing rate (52%). Improving TAAS 
scores was one of the main goals for implementing the ILS according to the principals responding the 
Principal Questionnaire. The emphasis on language skills may aid writing skills. However, other 
factors such as changes in educational services coincident with the implementation of the ILS may 
have impacted the result. 
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FIGURE 11 

1993-94 TAAS PASSING RATES FOR CCC, JOSTENS, AND ALL ILS 
ELEMENTARY SCHOOLS COMPARED WITH THE DISTRICT AVERAGE 
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Secondary ILS Schools 

Two of the secondary schools are using the CCC labs to help students who have not passed the TAAS 
tests. Figure 12 shows the percentage of secondary students passing the TAAS compared to the 
District average. 



FIGURE 12 

1993-94 TAAS PASSING RATES FOR SECONDARY ILS SCHOOLS 
COMPARED WITH THE DISTRICT AVERAGE 



PROGRAM 


READING 


WRITING 


MATHEMATICS 


Passing Ail Tests 
Taken 


GRADE 


GRADE 


GRADE 


GRADE 




7 


8 


Exit 


7 


8 


Exit 


7 


8 


Exit 


7 


8 


Exit 


All Secondary ILS 

# StudcnU Tested 
% Fasaing 


76 
39% 


78 
45% 


92 
64% 


n/a 
n/a 


82 
40% 


92 
62% 


77 
16% 


78 
27% 


92 
33% 


81 
16% 


90 
18% 


92 
27% 


AISD 
% Passing 


63% 


69% 


75% 


n/a 


62% 


77% 


44% 


49% 


54% 


42% 


42% 


50% 
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All grades at the ILS secondary schools were below the District average for the percentage of 
students passing all TAAS tests. However, this comparison is not necessarily equitable as many of 
these secondary students are using the IJ-S programs because they have previously failed a section of 
the exit-level TAAS. A look at one high school that uses CCC for TAAS preparation shows that the 
ILS may help students who have previously failed the TAAS to pass the exit-level test. Figure 13 
shows the percent of secondary students passing the TAAS after participation in the TAAS 
preparation course using CCC. No numbers were available for the percentages of students who pass 
the TAAS on the second or third attempt; therefore, no comparisons could be drawn. 



FIGURE 13 

PERCENT OF SECONDARY STUDENTS PASSING THE TAAS AFTER PARTICIPATION 
IN THE TAAS PREPARATION COURSE USING CCC 



Date TAAS Taken 


Reading 


Mathematics 


Writing 


March 1994 


24 (37%) 


20 (22%) 


14 (28%) 


May 1994 


4 (80%) 


5 (83%) 


1 (33%) 



Secondary School Achievement Gains Using CCC Reports 

Robbins High School and Fulmore Middle School provided data showing individual student gains 
from the beginning of the program along with the amount of time spent on CCC during the year. 
In reading, the students averaged a gain of .02 months for every 15 minutes on the computer. 
In mathematics, students averaged a gain of .01 month, and in English as a Second Language 
(ESL), students averaged a .02 month gain for every IS minutes of computer instruction. 
Fifteen minutes was the typical length of time a CCC program runs at the secondary level. See 
Figure 14 for the maximum and minimum gains and the standard deviations of student gains. 

If achievement gains are directly proportional to the amount of time spent in the lab, substantial gains 
may be possible at the secondary level if CCC is implemented 15 minutes each day as designed. A 
gain of 4.0 grade equivalent in reading, and 2.1 grade equivalent in mathematics may be possible in 
one year. ESL students could gain 4.2 grade equivalent after using CCC for one year. However, the 
amount of achievement gain which could be attained by increasing the number of minutes per day is 
unknown. Twice the number of minutes may not produce twice the achievement gain. Since none of 
the AISD secondary schools exceeded their predicted gain with CCC, this prediction would need to be 
studied further. 

In addition, the reports' produced internally by CCC are based on a built-in assessment scale whose 
psychometric characteristics (reliability and validity) are not known. Clearly, the gains predicted by 
the CCC assessment system are not supported by the NAPT results. 
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FIGURE 14 

AVERAGE GAINS AND STANDARD DEVIATIONS FOR EVERY IS MINUTES OF 
ILS COMPUTER INSTRUCTION FROM A SAMPLE OF SECONDARY STUDENTS 



Subject Area 


Average Grade 
Equivalent Gain Per 15 
Minutes of Instruction 


Maxihtum 
Gain 


Minimum 
(5ain 


Staitdat><i 
Deviation 


Reading 


.023 


.225 


.004 


.026 1 


Mathematics 


.012 


.038 


.002 


.007 I 


ESL 


.024 


.194 


-.011 


.184 1 



Promotion Rate for Elementary ILS Schools 

Promotion rates at elementary ILS schools were below the District average of 97. 7% for 1993-94. A 
comparison of 1992-93 and l?;^3-94 promotion data was made to discover if the ILS had an impact 
on promotion at elementary schools using ILS. Promotion refers only to those students who have 
successfully completed their current grade, not students who are "placed" in the next grade. As 
shown in Figure 15, schools with Jostens (Brown and Sims) were the only campuses to show an 
increase in promotion rates from 1992-93 (95.7%) to 1993-94 (96.1%). 



FIGURE 15 

PROMOTION RATE BY ELEMENTARY ILS PROGRAM, 
1993-94 COMPARED WITH 1992-93 



PROGRAM 


1992^93 


1993-94 


differenCb 


Elementaiy ILS Students 


95.9% 


94.4% 


-1.5% 


Elementary CCC StudenU 


96.0% 


94.0% 


-2.0% 


Elementary Jostens 
Students 


95.7% 


96.1% 


+0.4% 


Chapter 1 ILS Students 


95.1% 


93.9% 


-1.2% 


Chapter 1 CCC StudenU 


94.9 


93.2% 


-1.7% 


AISD 


97.6% 


97.7% 


+0.1% 
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ILS OBSERVATIONS 
Instrument Design and Test 

Staff members observed students working at the CCC and Jostens systems at the nine Chapter 1 
campuses and the eight non-Chapter 1 campuses to determine time on task. A minute-by-minute 
observation with an additional 15-minute classroom taUy was the result. Trial runs with the 
observation scale were conducted by ORE evaluation staff at Mathews and Wooten Elementary 
Schools to assure interrater reliability. Minor changes were made to the scale before the final test. 
See Attachment D for a copy of the observation instrument. 

Procedure for Selecting Classes and Student<g to Be Observed 

Before determining who would be observed, it was necessary to investigate how each school used the 
ILS. Only the classrooms that had used the ILS for at least one year would be observed. Schedules 
for using the CCC and Jostens labs or distributive networks were requested from each of the 
classrooms to be observed. It was determined that Chapter 1 staff would observe at the Chapter 1 
classrooms and a local staff person would observe the non-Chapter 1 classrooms (Travis Heights 
Elementary School; Fulmore, Pearce, and Porter Middle Schools; and Crockett, Reagan, Robbins, 
and Travis High Schools). A total number of sessions per grade was tallied and a 10%-15% sample 
of grades served was targeted. 

Figure 16 shows the total number of ILS sessions for Chapter 1 classrooms and the targeted number 
of observations per grade level. 



FIGURE 16 
ILS SESSIONS PER WEEK BY GRADE, 
CHAPTER 1 SCHOOLS 



SCHOOL 


Pre-K 


K 


Isi 


and 


3rd 


4th 


5Ui 


Special 
Education 


Total 
SessSooa 


Allison 










24 


20 


20 




.64 


Brooke 








25 


15 


15 


15 




70 


Brown 








8 


12 


18 


8 




46 


Jordan 

(Diit. Network) 










15 


15 


15 




45 


Norman 




1 


9 


7 


8 


6 


6 


2 


45 


Oak Springs 










15 


15 


15 




45 


Pecan Springs 








5 


20 


25 


20 




70 


Sims 


3 


2 


2 


3 


10 


15 


10 


2 


47 


Winn 




5 




5 


25 


25 






60 


ToUl Sessions by 
Grade 


3 


14 


U 


53 


144 


154 


.10^ 


4 


49a 
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Students were chosen on a random basis from the selected classes prior to the observers' visits to the 
campuses. A total of 63 observations was completed on the nine Chapter 1 schools, giving a 12.8% 
sample. The length of sessions varied from 12 minutes (Jordan) to 77 minutes (Travis High School). 
The average length of the ILS session was 30 minutes for all schools observed, and 27 minutes for 
Chapter 1 schools 



Observation Findings 

Students were observed for a total of 2,228 minutes while in the CCC or Jostens labs or distributive 
network (Jordan) at the Chapter 1 and non-Chapter i schools. Eighty-one percent of the time 
allotted to use of the ILS was used interacting with the computer on academic, technical, or 
procedural tasks. Descriptions of these tasks are found in Attachment D. Students needed 
assistance from the teacher or lab aide 12% of the time. 

While working with CCC or Jostens, most students were attentive to the task at hand. Only 5% of 
the time were students involved in off-task behavior. Non-instmctional time (transition, dead time, 
and waiting for the teacher) comprised 9% of the time. Figure 17 summarizes the data obtained from 
the observations. 



FIGURE 17 

PERCENTAGE OF 1 IME SPENT ON VARIOUS TASKS 
DURING ILS OBSERVATIONS 



VARIABLE 


PERCENTAGE OF TIME* 


Interaction with Computer 


81.4 


Academic Task 


80.7 


Tv^chnical Task 


7.0 


Procedural Task 


2.2 


Individual Interaction with Teacher 


6.3 


Group Interaction with Teacher 


5.5 


Interaction with Teacher-Total 


11.8 


On-Task Behavior 


88.9 


On-Task Interaction with Other Student 


4.1 


Off-Task Behavior 


4.8 


Non-Instructional Time 


8.8 



* V*ri*blet are combined for percenUgei shown. Nun»ber» will not cqu«l 100%. 
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A 15-minute tally of the entire classroom setting was made during each observation. This tally included 
1,707 students, most of whom were using the ILS. The average class size at all schools was 15 students. 
The 113 tallies showed 74% of the students observed in the classrooms were interacting with the 
computer. This percentage seems lower than expected, but the fact that at Jordan (distributive network) 
only one or two students at a time worked on the ILS while the rest of the class worked on classroom 
instruction must be taken into consideration. Eighty-nine percent of the students were engaged in on-task 
behavior during the 15-minute tallies. Non-instructional activities occurred 7% of the times for students. 
Figure 18 shows the same variables used above to illustrate the percentage of students engaged in each 
behavior during the 15-minute tally of classrooms. 



FIGURE 18 

PERCENTAGE OF STUDENTS ON VARIOUS TASKS 
DURING 15-MINLTE CLASSROOM TALLY 





PEIlCEkrAGE Of" STUDENTS AT 1 


Interaction with Computer 


74.0 1 


Academic Task 


76.9 


Technical Task 


2.3 


Procedural Task 


2.8 


Individual Interaction with Teacher 


3.0 


Group Interaction with Teacher 


4.9 


Interaction with Teacher-Total 


7.9 


On-Task Behavior 


89.1 


On-Task Interaction with Other Student 


7.2 


Off-Task Behavior 


3.7 


Non-Instructional Time 


7.0 



* Viriiblci ire combined for pcrccntagci ihown. Numbers will not equal 100%. 
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REFLECTIONS ON EFFECTIVENESS OF ILS 
The Role of the Principal 

The involvement of the principal in the selection, implementation, and utilization of the ILS is thought 
to be important to its success at the campus. Principals at ILS schools were surveyed in the spring 1994 
ORE Coordinated Survey. Of the 98 principals who responded to the statement, V have been supportive 
of the use of an integrated learning system at my campus," 72% (71) agreed or strongly agreed. When 
asked on the 1994 ORE Coordinated Survey if thck principal was supportive of the ILS technology at 
their school, 89% of the teachers at ILS schools responding agreed or strongly agreed. This suggests that 
both the principals and the teachers agree that there is support for the use of ILS at their campuses. 

Principals were asked to respond to questions about the utilization and implementation of the ILS at their 
campus. Their input was solicited to gain from their experience. Fifteen questionnaires were sent to 
elementary principals who current:.y have an ILS, and nine responded (a return rate of 60%). Of those 
responding, eight (89%) were pleased with the implementation and utilization of the ILS at the campus. 
However, Brown Elementary School experienced frequent technical difficulties and has chosen not to have 
the Jostens lab for the 1994-95 school year. See Attachment C for a copy of the Principal Questionnaire. 
Principals had the following suggestions for schools which are considering the installation of an ILS in 
the future: 

• Do not rely on one program to meet the needs of your slow learners. 

• Utilize all the features of the program, not just reading and mathematics. 

• Staff development is essential to the effectiveness of the ILS. 

• Make sure your school has the hardware needed to support an ILS. 

• Use reports to show and explain to parents in what area their child needs help. 

• Investigate all integrated learning systems before deciding on one. 

• Principal needs to be highly literate in technology to evaluate options in relation to the needs and 
priorities of their school. 

• Have a lab aide with CCC-it will prove to be the major difference between another supplemental 
program and a tremendous additional asset and instructional resource. 

• Watch each cost line-some items are not necessary. 

• Ensure that the District has purchased a full maintenance service contract-paid by AISD. 

• CCC is the better investment for the limited budget. 

• Customize the program to the needs of your students and staff. 

• Pay close attention to the software and hardware compatibility. 

• Start planning early. 

• Monitor, monitor, monitor! 

The Role of the Teacher 

Teachers at ILS schools received varying amounts of training. Each teacher was supposed to get one day 
of training. At schools where there was no lab aide, a contact person for the school received more 
extensive training. 
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Several teachers told ORE observation staff that they did not have enough training to use the system to 
its potential. Some teachers rarely received or used the reports generated by the ILS on student progress. 
In response to the statement -lam getting the training I need to use the ILS technology effectively- 65% 
of the elementary ILS teachers who responded agreed or strongly agreed, while 13.5% disagreed or 
strongly disagreed. The role of the teacher becomes even more important at schools without a lab aide. 

The Role of the Lab Aide 

Of the nine Chapter 1 schools studied, Brooke, Pecan Springs, and Jordan did not have lab aides. The 
ILS at schools with a lab aide seemed to run more smoothly. If there was a technical problem (and there 
were many during the observations), the lab aide could attend to it. At the schools without the aide, the 
teacher in tlie lab at the time of the problem would do what he or she could, but, if the problem was not 
corrected, the teacher would have to leave a message for the schoolwide ILS contact person who is also 
a classroom teacher. In response to the statement, "TJie person who most often assists my students with 
computer-assisted instruction is, " 63 % of the classroom teachers at elementary ILS schools indicated that 
they were the ones who most often assisted their students with the ILS. Only 20% of the teachers 
surveyed agreed that the person who most often assisted the student was a lab aide. 

The benefit of the lab aide is an issue each school should consider before implementing the ILS. There 
is a higher cost for the distributive network to allow for wiring throughout the school and the purchase 
of additional computers. In those schools without an aide, there is a contact person who has had more 
extensive training and deals with system problems. Teachers at the campuses must be trained to deal with 
the everyday use of the system. 

Many schools that use the lab setting use their existing computer lab which saves money on the hardware. 
A teaching assistant or lab aide may assist teachers by generating reports on student progress. The lab 
setting can be used by all students on campus while the distributive network in classrooms must focus 
on specific grade levels or needs. 

SUMMARY 

There is growing interest in the use of integrated learning systems as school districts are trying many 
alternative methods of instruction to meet the needs of academically diverse children. Twenty-two of the 
AISD schools used an ILS with the aim of improving student achievement in 1993-94. CCC and Jostens 
were the major integrated learning systems used in the District. 

Students were observed for a total of 2,228 minutes while in the CCC or Jostens labs or distributive 
network (Jordan) at Chapter 1 and non-Chapter 1 schools. Eigiity-one percent of the time allotted to use 
of the ILS was used interacting with the computer on academic, technical, or procedural tasks. While 
working with CCC or Jostens, most students were attentive to the task at' hand. Only 5% of the time 
were students involved in off-task behavior. 

Gains in student achievement have not been significant enough to declare the CCC or Jostens programs 
effective. The gains that have been made at some schools and grade levels warrant the continued use of 
the systems. Review of the suggestions made by principals about implementation and utilization of the 
ILS would be of value for schools considering an ILS. 
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Computer 


Classroom 


Computer Lab 


Administration 


IBM PC (Model 5150 & 5155) 








IBM PC XT (Model 5160) 








IBM PC XT286 








IBM PC AT 








IBM PS/2 Model 25 








IBM PS/2 Model 25 286 








IBM PS/2 Model 25 SX 








IBM PS/2 Model 30 286 








IBM PS/2 Models 35 SX & 35 SL 








IBM PS/2 Model L40 SX 








IBM PS/2 Model 40 SX 








IBM PS/2 Model 55 SX & 55 LS 








IBM PS/S Model S6 SX, S6 Lii, Sti 
SSLC & 56 SLC LS 








IBM PS/2 Model 57 SX 








IBM PS/2 Model 70 386 








IBM PS/2 Model P70 386 








IBM PS/2 P75 486 








IBM PS/2 Model 80 386 








IBM PS/2 90 XP 486 








IBM Model PS/2 95 XP 486 








IBM PS/2 Model 60 








IBM PS/2 Model 9585 








EduQuest 30 








EduQuest 40 








EduQuest 50 








Portable/Laptop 








Other 
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Computer Brand & Model 


Classroom 


Computer Lab 


Administration 
















































































■- — 
























: 








































, 
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Date: 

Completed By: 
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ComDUter 


Classroom 


Computer Lab 


Administration 


ADOle II & 11+ 








AoDle lie 








Annie Has 








Macintosh 512 & Plus 









Macintosh SE 








1 Macintosh Classic 1 








Macintosh Classic II 1 








r Macintosh Color Classic 1 








Macintosh SE/30 
















Macintosh LCI!! 








Macintosh LC 475 1 








Macintosh LC 520 i 








Macintosh LC 550 1 








Macintosh LC 575 1 








1 Macintosh lisi 1 

1 lW»**t^^t ■ • • mm^^w- 1 








Macintosh l!cx & llcl 1 








Macintosh llvx 








1 Marintocjh Centris 610 








Macintosh Centris B50 








1 Marintn^h II & \\X 








1 Msirintofih llfx 








1 MarintnQh Quadra 700 








1 MrtrintnQh Ouadra 300 

I IVlCtwl 1 llWwl 1 www 








1 Macintosh Quadra 900 








1 Marintnch Quadra 950 

1 mClwII wl 1 \JtUGiV4l Kt WWW 








1 K/ioointnch ^P^ntri^^ Quadra 660AV 








r Ms^pintnsh Quadra 840AV 

1 IflClwlllfcwwII ^iUCiWl M W-TW#^ W 








1 PntAffir Msicintofih 6100 








1 Power Macintosh 7100 








Power Macintosh 8100 








1 Apple Workgroup Server 60 








h" Apple Workgroup Server 80 








Apple Workgroup Server 95 








1 Macintosh Powerbook/Portable 








1 Other 
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School: 
Date: 
Completed By: 



Printers for IBM and compatible 



Printer 


Classroom 


Computer l^b 


Administration 


Daisy Wheel 








Dot Matrix 








Ink Jet/Bubble Jet 








Laser 









Printers for Apple and Macintosh 



Printer 


Classroom 


Computer Lab 


Administration 


Dot Matrix 








InlcJet/Bubbie Jet 








Laser 









Peripherals for Apple Macintosh & IBM PC's and Compatibles 



Peripheral 


Classroom 


Computer Lab 


Administration 


CD-ROM Player 








Laser Disc Player 








Scanner 








External Hard Drive 
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Attachment C 



Houston 

PRINCIPAL QUESTIONNAIRE 

1) What was the initial goal for the implementation of CCC or Jostens at your campus? 
How did you chose the students to be targeted to use the ILS (Integrated Learning 
System)? 



2) How do you feel about the implementation and utilization of the ILS at your campus? 
How did you chose between the lab setting and the distributive network in the 
classrooms? Will you change anything next year? 



3) What advice could you give to schools who are considering the installation of an ILS 
in the future? 
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Date: _^ 

Time: 

Student ID: 

Student Name: 

Classroom Teacher? Yes 



No 



ILS Observations 
Spring 1994 



Lab Aide? Yes 



No 



School: 

Grade: 

Observer: 



Other Adult? Yes 



No 
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ILS Observations 
Definitions of Terms 

Task (A) 

This stands for Task (Academic). Students falling in this category are those working in an 
academic assignment or receiving an academic presentation on the computer. 

In order for the task to be considered academic, the student must be reviewing old information 
or receiving new information from the computer about some skill involved in readmg, wnting, 
spelling, grammar, mathematics, science, etc., or be involved in solving problems/answermg 
questions, etc. 

This category does not include instructions from the teacher or the computer about activities 
which are preparatory to beginning an academic task, or m^^ssaiy for completing an academic 
task, such as those described under Task (T) or Task (?) below. It does include activities which 
are related to academic skills -- reading stories on tiie computer, playing computer games related 
to an academic subject, etc. 



Task (T) 

Students classified in tiiis category are engaged in some technical CO task related to tiie 
operation of tiie computer. 

Such tasks include turning tiie computer on or off, finding tiieir computer files/folders, moving 
tiirough transitions from one computer file/folder to anotiier, formatting a document to pnnt, 
printing a document, or trying to figure out how to get the computer to do something (I d hate 
to know how much of our time on the computer is spent doing tiiese kinds of tilings!) 



Task (P) 

Students classified in tiiis category are those who are clearly engaged in some procedural activity 
which is preparatory to beginning an academic activity, or is necessary for finishing it. Students 
are not expected to spend much time in this category, but examples of activities tiiat would be 
categorized here are getting out books or other materials, turning in work, or putUng headings 



on paper. 



Listn Teh (A) . u i i o 

Students in tiiis category are listening to the teacher give a presentation to tiie whole class or a 
grout) of students in tiie class on an academic subject. Some examples of tiiis are: tiie teacher 
lecturing to the class (or a group), teacher asking questions and responding to questions from 
the class (or the group), teacher giving a demonstration, teacher reading aloud, teacher checking 
work with tiie class (or the group). 
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Listn Tch (T) 

This is similar to the above category in terms of the teacher interacting with the whole class or 
a group of students in the class, except that the activities deal with computer routines and 
procedures. Examples of such activities are the teacher explaining to the whole class or a group 
of students how to get to a new file/program on the computer, how to print something, or which 
computer keys to push for a particular purpose. 

Listn Tch (P) 

Again, this is similar to the above two categories in that the teacher is interacting with the whole 
class or a group of students. To be categorized in this category, the student must be listening 
to the teacher discuss something procedural. 

Examples are explaining what the rules about the computer lab or student behaviors are, what 
will be the day's activity on the computer, how to get to an activity, when they are to move 
from one activity on the computer to another, when they are to quit computer work, and what 
to do with completed work. 

This format is probably used more frequently in the beginning of the year, or with students new 
to the school or the computer. 

Wa it Tch 

Students in this category are waiting for assistance from the teacher. They must have indicated 
(either through raising a hand, or through calling out for the teacher) that they need help to be 
counted in this box. 



Tch (A) 
Tch (T) 
Tch (P) 

These three categories are very similar in that they all refer to a student interacting individually 
with the teacher. They are different in the content of the interaction. Tch (A) refers to 
interactions about academic matters, Tch (T) refers to help or instructions from the teacher 
regarding technical matters, and Tch (P) is the category for coding any other interaction with 
the teacher, such as disciplinary interactions regarding the student's behavior. 



Stud (A) 
Stud (T) 
Stud (P) 
Stud (O) 

These four areas relate to the student interacting with other student(s). Stud (A) is for 
interactions in which the student seeks help or helps another student with an academic area, Stud 
(T) is for communications about technical matters (as in "how do you print this thing?"), 
Stud (P) is for procedural matters (as in "What did the teacher say we need to be working 
on?"), and Stud (O) is for any other interactions that are off-task. 
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Students are classified in this category when they are very clearly misbehaving and doing 
something which the teacher does not approve of. It is not essential that the teacher correct the 
students for them to be classified here. Examples of behaviors which would be classified here 
are: looking at someone else's screen when it's not allowed, playing around, daydreaming, 
getting into files/programs in the computer they're not supposed to, and visual wondenng. A 
student who is off-task because he/she is talking to another student should be classified in the 
Stud (O) category. 



This stands for Dead Time. Students should be classified here when the observer realizes that 
there is nothing specific which students are supposed to be doing and when they are not engaging 
in unsanctioned behavior. This would include students who are waiting for a transition as part 
of the whole class and students who have finished all of their assigned work and who have not 
been given anything else to do. 



Trans ... . . ... , 

This category should be used when students are changing activities. Most hkely this will only 
occur when they first enter the lab until they are settied in tiieir seats, and when they are getting 
ready to leave. 



Comment 

Write the topic or academic area (reading, mathematics, science, etc.) students are supposed to 
be working on in this box (even if the student is currentiy off task). Write down the academic 
area in the first observation period, and then again when it changes (as long as you do not 
indicate a new academic area, it will be assumed that it has not changed). 

You may also jot down any other comments that may help you remember what was going on 
in the class at the time (for your narratives). 
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